Chalcogenide optomemristors for multi-factor neuromorphic computation

Neuromorphic hardware that emulates biological computations is a key driver of progress in AI. For example, memristive technologies, including chalcogenide-based in-memory computing concepts, have been employed to dramatically accelerate and increase the efficiency of basic neural operations. However, powerful mechanisms such as reinforcement learning and dendritic computation require more advanced device operations involving multiple interacting signals. Here we show that nano-scaled films of chalcogenide semiconductors can perform such multi-factor in-memory computation where their tunable electronic and optical properties are jointly exploited. We demonstrate that ultrathin photoactive cavities of Ge-doped Selenide can emulate synapses with three-factor neo-Hebbian plasticity and dendrites with shunting inhibition. We apply these properties to solve a maze game through on-device reinforcement learning, as well as to provide a single-neuron solution to linearly inseparable XOR implementation.


Supplementary
.3. (A) (i) False-colored SEM micrograph of an Ag/GeSe3 stack. Under an applied electric field using tungsten probes, the device undergoes non-volatile switching, which is preceded by the formation of a dendritic filament. (ii) Inset shows a composite like the structure of the stack wherein nanostructures are embedded in the GeSe3 film. (iii) The bottom inset shows the zoomedin region of a filament; the yellow line is the elemental scanning line. The right panel is EDX elemental maps of the different elements constituting the device, illustrating the filament to be rich in Ag, Ge, and Se. (B) Electron diffraction patterns of the filament and matrix, showing that the filament is crystalline and the matrix is amorphous. Figure S1.4. (a) Energy-dispersive X-ray spectra of a GeSe3 thin film. (B) Raman spectra of a GeSe 3 thin film, Ag/GeSe 3 stack and conductive filament. The 194cm −1 is the ETH vibration modes of the corner-sharing GeSe4/2 tetrahedrons and the 211cm −1 peak represents the ES breathing vibrations of the edge-shared Ge2Se8/2bi-tetrahedrons. The peak at 262cm −1 is the Se-Se bonds and indicates the configuration of Se is in the form of Se-8 rings. At the filament, the ES, Se-Se and Ge-Ge peaks are suppressed; highlighting a change in the local bonding configuration of the atoms. This observation is in line with previous reports and are suggestive of depletion of Se chains in the matrix via photoactivated reactions between Ag and Se (Journal of Physics and Chemistry of Solids 68, 866-872 (2007)). Note that the Raman measurements were carried out using 639 nm optical excitation and at low optical power (1 mW/3s integration time and 3 accumulation cycles) to avoid photo-induced changes. (c-d) Raman spectra of GeSe3/Ag 36 months after deposition. We would like to point out that our EDX and microanalysis results, both on the chalcogenide sputtering target and the deposited thin fim suggest GeSe3 as the composition. However, following reference Journal of Physics and Chemistry of Solids 68, 866-872 (2007), the Raman spectra of the films suggest a rather higher concentration of Se, giving a composition close to Ge15Se85. However, in our experiments, we have found that optical exposure during laser exposure in the Raman measurements can induce structural changes to the films. For purpose of clarity, we choose to represent our film with the composition GeSe3. It is in the interest of our future report to more carefully study the film composition with other analysis methods. Figure S1.5. (a) X-ray diffraction of spectra (black trace) of a 100 nm GeSe3 thin film. The film is amorphous with no characteristic peaks. (b) X-ray diffraction of spectra of a 1µm GeSe3 thin film at a grazing angle annealed at 330 0 C for 6 mins in room conditions. No peak corresponding to crystallites is observed. (c) A scanning electron micrograph of the filament in an Ag(15nm)/GeSe3(38nm) stack. Inset is a zoomed-in view of the bottom branch. Note that around the periphery of the filament, the matrix is devoid of the nanostructures, which are otherwise homogeneously distributed in the matrix.

Supplementary
Supplementary Figure S1.6. An optical micrograph of a Pt/GeSe3/Ag device after 36 months from deposition. (b) An optical micrograph of a Pt/GeSe3/Pt device after 36 months from deposition. (c) It is noted that Ag diffuses into the chalcogenide film (which acts asolid-electrolyte), and forms into embedded globules like structure. Pt however is not noted to diffuse. Inset shows an atomic force microscopy line scan on GeSe3/Ag pad. (d) An atomic force microscopy map of (c) showing photodiffused Ag rich nanostructures. Ag diffuses into the film through photo-chemical processes (Journal of non-crystalline solids 124, 186-193 (1990), Journal of Physics and Chemistry of Solids 68, 866-872 (2007), Thin solid films 449, 248-253 (2004), Advances in Physics40, 625-684 (1991)), the extent to which is governed by the saturation limit of Ag doping, which inturnis governed by chalcodgenide electrolyte film (Journal of Physics and Chemistry of Solids 68, 866-872 (2007)). Figure S1.7. Energy-dispersive X-ray spectroscopy on an Ag/GeSe3/Ag stack (top to bottom panels) using transmission electron microscopy operated in the STEM mode. The entire top Ag layer and a major portion of the GeSe3 film are sputtered away during focussed ion beam based sample preparation (evidenced by the Ga content in the films). However, the more interesting region is the interface between the bottom Ag and GeSe3, which highlights the diffusion of Ag into the GeSe3 film. This is a result of the spontaneous intermixing of Ag into GeSe3 volume. Figure S1.8. Cyclic current (10 y )-voltage characteristics of Ag/GeSe3/Ag stacks of crosssectional area (a) 49.5µm × 49.2µm, (c) 25.3µm × 24.8µm, (d) 15.2µm × 14.5µm, (d) 5µm × 4.8µm. (e) Resistance vs device cross-sectional area of the devices in both their HRS and LRS states. In a filamentary memristor, the LRS state can be regarded as a parallel configuration of the conductive filament and the rest of the device. Thus, when the device area is changed (the bottom and top electrodes), the resistance of the HRS state (devoid of a filament) is expected to decrease from the inverse relationship of resistance with the area, but the resistance of the LRS state is expected to not change since the dimensions of the filament across which the voltage drop do not change.

Supplementary
Supplementary Figure S1.8. A 300 nm thin SiO2 is thermally grown on an n-Si (100) wafer using wet oxidation at 1050 0 C. (b-c) The wafer is coated with an S1813 positive resist for patterning the bottom electrodes (BE) using photolithography (2 mins baking at 120 0 C, 8-20 seconds exposure time/30-60 seconds, development in MIBK developer/30-60 seconds reaction termination in isopropanol). (d) After pattering, the SiO2 on the wafer is reactive ion etched in the areas exposed by UV exposure (to take the shape of the BE) using the chemistry CHF3=50 sccm, Ar=10 sccm, O2=2 sccm, Power=100 W (at etching rate 21 nm/min). The etch depth is carefully optimized to match the BE thickness so that BE is planarized with respect to the SiO 2 surface. (e) Based on the stack type and using RF sputtering the BE is deposited in Ar atmosphere at a working pressure of 3.6e-3 mtorr. 2-3 nm of sputter-deposited Ta was used as an adhesion layer between BE and SiO2. Sputter conditions are: Ag (Power=30W/ rate=4.5 nm/min) or Pt (Power=40W/ rate = 4.2 nm/min) or ITO (Power=30W/ rate = 3.5 nm/min) and Ta (Power=120W/ rate = 4.6 nm/min). Following this, the metals were lifted off in an acetone solution, placed in a water bath at 65 0 C. (f) Second photolithography using S1813 positive resist is then performed (2 mins baking at 120 0 C, 8-20 seconds exposure time/30-60 seconds) to pattern the top electrode (TE) and chalcogenide GeSe3 (Ch). Global alignment markers are used to align the TE with BE. (g-h) After development (in MIBK developer/30-60 seconds reaction termination in isopropanol), GeSe3 film and TE (Ag, ITO, Pt) are sputter deposited. Sputter conditions for GeSe3 are Power=30W/ rate = 4.9 nm/min under working pressure of 3.6e-3 mtorr. (i) The depositions were then lifted off in an acetone solution, placed in a water bath at 65 0 C, and the wafers were then used for testing. An important fabrication step we note is the requirement to not overfill the trench with BE. If the BE is overfilled and post chemical mechanical polishing is not carried out, then BE and TE are noted to be electrically shorted, making devices electrically conductive and unusable. (j) In our experiments the optical illumination is always performed from the top, such that light traverses from the top electrode toward the BE. Illumination itself is performed using a laser beam to avoid stray exposure. The observation of a short-circuit current indicates that our devices work with a photovoltaic mode of operation, governed by asymmetric Schottky barriers between GeSe 3 layer and the bottom and top electrodes. This in turn implies that an in-built electric field (Ein-built) exists in the GeSe3 layer. (b) Under increasing optical illumination we also find that the open-circuit voltage shifts, which further suggests that the Schottky barriers -thus the in-built electric field-change under illumination. (c) When an external voltage is applied to the channel, an external electric field (External) is set in the GeSe3 layer. If ETH is the threshold electric field required to switch the device from HRS to LRS, then the voltage (VTH) required is reduced if both the in-built and external field point towards the favorable filamentation direction. (d) When Ein-built and External are directed in the opposite directions the switching voltage is expected to be larger. Sketches (c) and (d) point to our experimental observations of increasing and decreasing (VTH) for different external voltage polarities. We however note that while our device functions in a photovoltaic mode, the photovoltaic mode does not exclusively render the observed optomemristive switching characteristics. Chalcogenide glasses are a good solid electrolyte for Ag. Furthermore, a unique feature of chalcogenide glasses GexSy and GexSey is their accelerated diffusion for Ag under the action of light (Journal of non-crystalline solids 124, 186-193 (1990), Journal of Physics and Chemistry of Solids 68, 866-872 (2007), Thin solid films 449, 248-253 (2004)). It is understood that under illumination an electrical potential is created via photochemical action, and the resultant electric field provides sufficient energy for Ag cations to diffuse from the interface into the bulk of the chalcogenide film (Thin solid films 449, 248-253 (2004)). Thus, in effect illumination provides an additional knob (beyond electrical voltage) to modify the filamentation dynamics of the devices. Note that photodoping/photodiffusion occurs readily (at a slower rate) in our devices when they are exposed to ambient light. We discussed this in Figure S1.3. In Figure S1.6 we show photodiffusion in our films after 36 months from fabrication. Also note that optical heating inside the active area of the device can modify the equilibrium potentials, aiding the photo-diffusion effects. However, estimating the temperature rise in the chalcogenide film in the active area of the device is non-trivial since this requires an estimation of the optical properties and distribution of the photo-diffused and formed crystal in the active region of the device. We also find that for the used programming conditions, the conductance of the filaments in the LRS state does not exhibit an integer of the conductance quantum, which is likely due to scattering and imperfect contacts (Physical Review Letters 92, 106804 (2004), Nanoscale research letters10, 1-30 (2015)). Therefore, the conductivity in our devices is likely because of quantum mechanical tunneling (Nano Letters16, 709-714 (2016)).

Supplementary
We have simulated optical heating effects in our devices using COMSOL® Multiphysics for estimating the temperature rise in the films. Our model consists of two parts: a 2-D approximation that models the layers of different materials that formed the nano-device. By simulating the way an incident electromagnetic wave of the same wavelength as the laser beam propagated through the geometry, it is possible to quantify the energy losses due to absorption within the device. The second part of the model is a 3-D heat transfer simulation that represented the geometry of the crossbar nano-device. The heat source utilized was calculated based on the energy losses obtained from the first part, therefore assuming that the energy lost due to optical absorption was transformed into heat and dissipated through the device geometry. We however note that the extinction coefficient of GeSe3 for the investigated wavelength is not significant. In Figure S2.4a we plot the absorption in GeSe3 when configured for different stack types. Note that GeSe3 minimally absorbs the incident electromagnetic wave (see Figure S2.4b). These results, however, assume that GeSe3 and Ag are physically separate layers with a sharp GeSe3/Ag interface. In real world, we have observed Ag diffusion into GeSe3, which should alter optical heating in the layer. To account for this, we perform our simulations for higher absorption (see Figure S2.4d(i)) in GeSe3. Figure S2.4b shows the 3D geometry of our crossbar device model. The temperature distribution across the cross-bar structure, obtained by applying 1 mW (same as in experiments) of optical power to the active region of the crossbar is shown in Figure S2.4d(ii). Figure S2.4e plots the cross-section of Figure S2.4d(ii): the interfaces between GeSe3 and Ag electrodes are in the Z direction. In Figure S2.4e(i) it can be seen that a thermal gradient is formed within the device due to heat loss via the Ag electrodes. For Ag we use thermal conductivity value of 429 W/mK and for GeSe3 0.53 W/mK. Note that the top electrode is optically heated which diminishes its role as an effective thermal sink and the bottom electrode is more effective in dissapting heat. The central region of the device reaches the peak temperature, but it is clear that the temperature rise in the chalclgenide film is minimal owing to the effective thermal sink provided by the Ag electrodes and only modest optical absorption in GeSe3. In Figure S2.4e(ii-iv) we repeat our simulations for a larger incident optical powers (5 mW, 10 mW, 65 W), which indicate the increase in the peak temperature due to greater optical heating. We however note these models need further experimental validation, and it is in the interest of future work to collaborate experimental and simulations findings on different chalcogenides systems to more exclusively delineate the optical heating and photochemical effects in the memristive switching behavior of devices.

Supplementary Section S3
Optical Cavity Simulation: Optical simulations were performed using a transfer matrix approach. The approach is analogous to a scattering matrix that relates the initial state and the final state of a physical system (electric fields) undergoing a scattering process. The refractive indices required for estimating the Frensel coefficients in this approach were obtained experimentally using ellipsometry.

Supplementary Section S6
In our demonstrations, the total power consumption in implementing three-factor RL equals ET=N×Ee+×Eo, where N is the number of synapses, Ee is the energy in the electrical signal and Eo is energy in the optical signal. Assuming the mouse takes a new step every other second, a total of t=5 secs are spent in finding the cheese during the course of which in our implementation the eligibility traces are flagged for 3 s. Thus, Eo=s×P0 mW, while Ee=N×500ns×Pe, where P0=0.13 mW, and P0=0.5μA×0.4VmW. ET=0.39mJ+4×10fJ=≈0.39 mJ. Thus, the energy dominating stimuli is the optical energy, which can be potentially lowered with the engineering of the resonating cavities, smaller exposure times, and the use of more absorptive chalcogenide materials. Compared to emerging memristive hardware approaches (2nd IEEEInternational Conference on Artificial Intelligence Circuits and Systems (AICAS), 218-222 (IEEE, 2020)), there is an order of reduction in energy consumption with the current unoptimized devices. Compared to CMOS, besides a small device footprint, the implementation of the three-factor learning rules with eligibility traces per synapse does not require complex memory structures for and wiring. In contrast, both computation and weight update occur in-place through multi-factorial computation. Compared to the memristive (phase-change memory) approach, the weight update process does not require read-verify cycles, as we employ the physics of optoelectronics to in place combine reward and eligibility traces. Besides, the time scales for computation are tunable in our devices to a wide degree. These, however, are fixed in PCMs thus limiting the realization of vastly different learning applications (2021 IEEE International Symposium on Circuits and Systems (ISCAS) 1-5 (2021). doi:10.1109/ISCAS51556.2021.9401446). We however wish to emphasize our motivation is to demonstrate use cases of optomemristors for some higher-order neuronal processes. Within the purview of RL, the implementation of the three-factor learning rules with eligibility trace per synapse in CMOS requires complex memory structures for keeping track of the eligibility trace and the weight. It is the interest of future work to scale up the approach. A discussion on how this could be done is provided in Figure S6. Figure S6. (a) When discussing scaling-up, individual optomemristive devices can be configured into crossbar topologies and be provided with supporting peripheral circuitries. D/A are digital to analog converters for input electrical and optical signals and A/D are analog to digital converters for the current sense. (b) In a crossbar of n by m optomemristive devices (synapses) each device is placed at the crosspoint between word (WL) and bit (BL) lines. The read and write voltage signals (Vn) are applied along the rows (WL) of the crossbar, and the current is measured across the columns (BL). For a given Vn, the current is a function of the non-volatile device conductance (Gn,m) that can be modulated using both electrical and optical write operation. These states represent the non-volatile states, and for reinforcement learning, for example, these are adjusted only during the training phase. (c) An illustration of the free-space setup for electro-optical modulation of synaptic weights for three-factor learning. Here, the optomemristive devices are electrically stimulated using on-chip circuitries, but optically using free-space lasers. In three-factor learning, for example, since at a given time single or a group of devices on the crossbar must see optical exposure the setup is provided with a digital micromirror device (DMD), that can expand and project a collimated laser beam (Practical holography XIII, vol. 3637,12-20 (International Society for Optics and Photonics, 1999)) of the desired pattern and orientation onto the chip. The DMD is controlled by a standard computer that communicates with the crossbar microcontroller and redirects the beam only to selected optomemristive devices. (d) A fully integrated on-chip architecture utilizing three-dimensional micro-LEDs laid in a crossbar topology. In this approach, the selectivity to expose only selected or a group of devices to light is established by fabricating micro-LEDs that emit distinct wavelengths (Nature Nanotechnology1-6 (2021)), and correspondingly fabricating optomemristive cavities that sense distinct wavelengths using stack thickness engineering. Here, each column (BL) of the micro-LED crossbar plane comprises LEDs that emit distinct wavelengths, i.e. all devices in the same column emit the wavelength of light and aligned on the top plane are optomemristive devices, which on the same column (BL) of their crossbar sense the same wavelength of light. Thus to trigger different columns of the crossbar, corresponding micro-LEDS columns must be turned on. (e) So far we discussed optomemristive devices which used optical signals for eligibility traces and electrical signals for a reward for three-factor learning. In an alternate approach, the electrical signal can be used as an eligibility trace and vice versa. One approach is to use the semiconductivity of the chalcogenide glasses by using a three-terminal configuration for the optomemristors. Here the temporal modifications to the conductance at the level of individual synapses are achieved using the gate signal (V Gate ), which is applied to the synapses using the diagonal selector lines (SL). A gate selector line modifies the Fermi level in the chalcogenide channel and through it modifies the temporal conductance. Note that the diagonal connections allow for parallel modulations, such that multiple devices can be modified simultaneously. In an integrated setup, the reward signal can be applied using single micro-LEDs or an array of micro-LEDs, all emitting light of the same wavelength. Only devices that are electrically stimulated using SL lines will undergo a switching event. This approach however requires that the chalcogenide can be electrostatically tuned, thus requiring the need for low band-gap and low carrier concentration materials.